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ABSTRACT 

The purpose of this study was to develop a taxonomy 
which would categorize high level physics problem-solving behaviors, 
and to examine the usefulness of such a classification system. This 
classification of learning objectives is based on complexity, a 
nonarbitrary measure which does not rely upon comparison between 
students but rather is based on the number of computational steps 
required. An experiment was conducted to compare the results of 
criterion referenced items based on the classification scheme with 
problem difficulty, significant differences indicated that 
performance on more complex problems could not be predicted on the 
basis of zero or one step problems. Included are samples of 
zero-step, one step and multiple step criterion test items. The 
computer programs used to analyze the data are also included. The 
project was carried out by the New York institute of Technology and 
was funded by the U. S. Office of Education. (TS) 
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Writing learning objectives for high l}* quantitative hard science 
courses requires careful apecif ieation of an elusive measure of level. 
For example 5 high school and college physics courses may cover essen- 
tially the same topical materials but with a profound difference in 
requirements and expected performance. The purpose of this study was 
to develop a taxonomy which would categorize high level physics prob- 
lem-solving behaviors, and to examine the usefulness of such a class!— 
fixation system. 

Our classification of learning objectives is based on complexity, 
a nonarbitrary measure which does not rely upon comparison between 
students. Basically, the scheme is to count the number of computa- 
tional steps required. We examine the consistency between the pro- 
posed classification of complexity and the more familiar measure of 
student difficulty. By using the system to determine the signifi- 
cance of complexify as a variable in absolute and relative student 
performance, we intend to investigate the relationships between 
student performance and levels of complexity* 

The classification system, as presented in this paper, evolved 
as a pragmatic consequence of an attempt to categorize the objectives 
of a physics program developed by educators for the U . S. Naval 
Academy. Originally the objectives were defined by verbal descriptions 
of the intended behavior, but various difficulties indicated a need to 
present objectives in more precise terms, as discussed in Section II. 
Objectives were therefore translated into problem form. The effort 
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required for classifying was thus reduced to one of categorizing 
physics problems * 

Once the objectives were defined in problem form, a classification 
scheme had to be developed. An objective measure of problem complexity, 
the number of computational steps required to solve the problem, was 
adopted,. n A taxonomy must be validated by demonstrating its consis- 
tency with the theoretical views in research findings of the field it 
attempts to ordar^ 1 ^ A general discussion of taxonomies is presented 
in Section III. 

The classification schema proposed in Section IV fits the defini- 
tion of taxonomy, as determined empirically* This procedure of 
counting mathematical steps is objective and a majority of physics 
questions fall uniquely and unambiguously into one of three categories. 
These categories have a clear hierarchal structure and can be assigned 
a numerical measure so that complexity can be viewed as an independent 
variable in student performance. 

We tested the classification system at the Naval Academy. The 
subjects in the experiment were sophomore midshipman, well above 
average in verbal and mathematical skill. Section V describes the 
experiment . 

The results and conclusions are discussed in Section VI, and 
implications are drawn in Section VII. 

^Benjamin 5. Bloom, J, Thomas Hastings, and George F. Madaus , 
Handbook on Fo rmative and Summative Evaluation of Student Learning 
(New York, 1971), p. 17. 
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The need for defining objectives cannot be overemphasized, 
Leonard Blackman , in Frontiers in Education , feels * "The development 
of education as a science rather than an art form will be wholly 
dependent on concrete and measurable statements of educational objec- 
tives coupled with the highly specific delineation of instructional 
procedures leading toward those ob ject ives . ft ^ Creating and construc- 
ting a detailed and comprehensive set of behavioral objectives is, 
then, a first major step in the process of course development The 
objectives, taken collectively, define the content, scope, and organ- 
ization of the course. They specify for both the student and the 
teacher the content and processes to be learned, and indicate the 
intended behavior of the student upon completion o r the course. 

In addition to defining the course, the objectives provide the 
course instructor or designer with a basis for evaluating student 
performance and course materials, and a basis upon which revision 
processes can be based. In a recent handbook by Benjamin Bloom, 
Leopold Klopfer promoted another use for objectives. 

In numerous science evaluation situations when observations 
of a student's performance are called for, there is a substan- 
tial degree of uncertainty in deciding how well he has done 
and how to guide him to do better. , , . The problem, then, 

is to find ways of developing much more detailed and precise 
specifications than have heretofore been attempted of the 
behaviors that the student is to attain. These specifications 



2 n The f Brave New World' of Special Education , M ed , Morey R, Fields 
(New York, 1967), p, 17. 
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would also delineate the prerequisite behaviors leading to the 
desired criterion behaviors, so that the student who has not 
attained mastery may be given soundly based guidance . 3 

Because the objectives "identify the kind of behavior that will 

be accepted as evidence that the learner has achieved thein,"‘ + we can 

determine exactly how much of what was taught was actually learned. 

Data obtained by analysis of test items related to the objectives can 

provide us with information as to which objectives have not been 

successfully attained. 

Still more can be provided by well defined objectives. They per- 
form an important organizational task for the student. As "advance 
organizers" they provide a label under which the learner can store new 
knowledge and concepts. Too, they can work as synthesizing agents, 
relating separate items to form a larger whole, "The usefulness of a 
structure for learning has to do with the ability of students , , . to 
use it as an organizing factor in their learning. 

Much of Robert Mager’s work has attempted to impress the educa- 
tional community with the need to define objectives specifically, A 
meaningful objective is one that communicates to the reader the writer’s 
instructional intent; the best statement is one that excludes the 
greatest number of possible alternatives, which succeeds in describing 



^"Evaluation of Learning in Science," Handbook on Formative and 
Summative Evaluation of Student Learning , Benjamin S, Bloom, et al. 
(New York, 1971) , pp. 637-638. 

^Barbara E, Schure , "The Computer: An Adaptive Teacher," Some 

Essays on Computers in Education , ed, Margaret E. Pineus (Cambridge, 
Mass,, Spring 1967), p, 51. 

^Bloom, Handbook, p. 12, 
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the terminal behavior of the learner well enough to preclude mi .^-inter- 
pretation. Others agree with Hager's requirement that the objectives 
be specific, 

A statement of an objective is an attempt by the teacher or 
curriculum maker to clarify within his own mind or communicate 
to others the sought-for changes in the learner. To accomplish 
this, the educator must choose words that convey the same 
meaning to all intended readers. Statements of objectives that 
can be interpreted differently by different readers give them no 
direction in selecting materials, organizing content, and 
describing obtained outcomes, nor do they provide a common basis 
for instruction or evaluation, 6 

The definition of an objective should include three facets. First, 

it must specify the behavior that a learner must elicit, preferably in 

terms that are understandable to both the Instructor and the learner. 

Because we cannot see learning directly, we must base our analysis of 

learning on something more evident. "The change in performance is what 

leads to the conclusion that learning has occurred.” 7 

The short-term objectives must be stated in an unambiguous way 
so that they are clear not only to the teacher himself .... 

In order to communicate objectives precisely and unambiguously, 
it is not enough to specify independently the content to be 
covered or the abilities and skills the student is expected to 
acquire. Communicability requires accurate statements of the 
expected behavioral changes related to a particular content area. 

Second, the conditions under which this learning is to be 

demonstrated should be specified. If textbooks, slide rules, and other 

information sources can be consul ted, the objective should so state. If 

the objective is to be achieved within a time limit, or if any other 

operating constraints are imposed, these should also be specified. 

®Bloom, Handbook , p, 20, 

7 Robert H. Gagne, The Conditions of Learning (New York, 1965), p. 6. 
®Bloom, Handbook , p, 23, 
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The criteria of acceptable performance is another aspect which 

should be included in the process of defining objectives. The student 

and instructor must both know what degree of competency is required 

and what measurable evidence of this achievement will be acceptable. 

One wav of making the objectives more detailed is to specify 
the behaviors the student should possess or exhibit if he has 
attained the objectives. . . . Another way of giving further 

clarity to the specifications of outcomes is to represent them 
in the form of the problems, questions, tasks, and the like 
which the student should be able to do or the kinds of reactions 
he should give to specific questions or situations. 9 

By developing criterion test items to clarify acceptable behavior 
at the time the objectives are defined 9 we have a basis for measuring 
the achievement of an objective, and, at the same time we avoid the 
possibility of making the objectives depend on relative performance. 
Measurement of achievement, Branson emphasizes, must be on an abso- 
lute standard, "since each person in the class has the opportunity to 
meet the prescribed specifications, regardless of how well or poorly 
his classmates do. Relative achievement of one student against 
another defers establishment of levels of acceptable competence until 
we grade student papers. 

Thus we build the term "Measurable Behavioral Objective" (MBO) , 
Since not all objectives are equivalent in importance, scope, or 
complexity, we can group these MBQs into terminal and enabling objec- 
tives. Terminal objectives are the desired final behaviors and ena- 
bling objectives are the more specific smaller blocks leading toward 
the terminal objectives. 

®Bloom, Handbook , p. 15. 

-^Robert K. Branson, "The Criterion Problem in Programmed 
Instruction," Educational Technology , July 1970, 36. 

O 
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Enabling objectives represent the transitional type of skill or 

knowledge which is believed to be a precondition for success on the 
terminal objective* Not all the enabling objectives are of equal 
importance, nor do we assume they are of equal degree of difficulty 
to attain* Enabling objectives may or may tot "build" upon each other; 
it is sufficient only that they facilitate the mastery of their asso- 
ciated terminal objectives. We have* then, distinct simpler behaviors 
as components of more complex behaviors. ^ 

Intuition suggests that not all learning is the same, and obser- 
vation and research concur. M A serious attempt to describe learning 
must take all these varieties into account. Naturally, it mast make 
differentiations among them, and classifications of them, if these 
are possible . What we attempt to do with defined objectives, then, 

is "to order phenomena in ways which will reveal some of their essen- 

1 1 

tlal properties as well as the interrelationships among them,” 

In the physics project, we began with verbal statements of objec- 
tives. Their lack of precision, however, resulted in a lack of 
clarity. Students were not able to assess the objective's level and 
scope, and so were not certain exactly vhat was expected of them. 
Professors with little specific classroom experience, regardless of 
their subject matter expertise, could not be sure to what extent any 
content was to be covered. Although there was professorial consensus 

■^Benjamin 3, Bloom, et al. , Taxonomy of Educational Objectives ; 
The Classification of Educational Goals . Handbook I : The Cognitive 

Domain (New York, 1956), p. 16. 

^Gagne , Conditions , p . 20 . 

13 

-Bloom, Taxonomy , p. 17. 
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on a list of objectives, there was little agreement on whether the 
objectives were achieved: evaluation of student progress was less 

than standardized. 

Use of verbal objectives revealed another more serious, though 
less obvious, hazard. Enabling objectives are clearly discerned by 
the course instructor or designer only when hi executes the behavior 
called for in the terminal objective. Each necessary step toward 
terminal behavior is then, by definition, an enabling objective. 

When a terminal objective is "fuzzy" the requisite steps for its 
achievement are even less clear. Our experience has been that this 
situation leads to poor ordering of enabling objectives under a 
terminal objective, a number of redundant enabling objectives, and 
a few omissions of enabling objectives. 

We reformulated the objectives into problem form to eliminate 
these difficulties. With a precise problem to represent a terminal 
objective, any subject matte-r expert can perform the steps for its 
solution and identify the individual steps as enabling objectives. 
Moreover, specifying objectives in problem form indicates to stu- 
dents what behavior will be considered acceptable and under what 
constraints their behavior must be evidenced. 

When acceptable performance is defined as correctly solving a 
problem, relative performance measures are obviated. Relative 
grades depend not on success in achieving the objectives but on 
achieving more or fewer than other students. 

We have become accustomed to classifying students in about 
five levels of performance and assigning grades in some 
relative fashion. It matters not that the failures of one 
year performed at about the same level as the C students of 
another year. Nor does it matter that the A students of one 

O 
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school do about as well as the F students of another, . , . 

There is nothing sacred about the normal curve. It is the 
distribution most appropriate to chance and random activity. 
Education is a purposeful activity , and we seek to have the 
students earn what we have to teach. If we are effective 
in our in true t ion , the distribution of achievement should 
be very different from the normal curve. In fact * we may 
even insist Lhat our educational efforts have been unsuc- 
cessful to the extent that the distribution of achievement 
approximates the normal distribution,^ 

The concept of casting objectives into problem form follows closely 
from developing test questions with which to measure achievement of 
those objectives. 

As objectives are developed and approved by the subject matter 
expert, L ^ t questions will be developed to cover these 
objectives, , , , In this way, one can determine from test 

items missed which educational objectives are not being met.^ 

By making the test items synonymous with the objectives we have an 
indication of attainment of the objectives which is more closely rela- 
ted to the objective. f, This represents the most detailed and precise 
definition , , , since it indicates the tasks the student is expected 
to perform and the specific behavior he is expected to exhibit, 



^^Bloom, Handbook , p. 45. 

-^Donald T, Tosti, Research and Evaluation Flans , Fart I 
(Wes t inghous e Learning Corporation, Palo Alto, Calif,, January 1969), 

p, 12, 

■^Bloom, T axonomy , pp, 44-45, 
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There is no definitive system for classifying learning behaviors 

in a way which is suitable to all fields and levels of learning. Nor 

do the experts— -educators , researchers, subject matter specialists — 

suggest that an ultimate model is necessary. Instead, in an attempt 

to classify learning conditions , general guidelines have been advanced 

which can be altered to suit specific courses of study. 

Generally accepted is the notion that we store information in a 

hierarchal manner, building upon simple facts and skills to develop 

more complex principles and concepts. Glassification of behaviors 

into terminal and enabling forms implies a recognition that some 

behaviors are subordinate to others. 

The clearest evidence that problem solving "can be conceived 
as a linear sequence of operations" is provided by the exten- 
sive work on simulation of cognitive -processes , , , , This 
same work also provides clear evidence of the importance of 
hierarchal relations not only in problem . jiving but in other 
kinds of behavior. 17 

From that point of concurrence, independently d » led taxonomies 
often vary, some based on difficulty, some on levels Oj. abstractness, 
others on still other variables. This is no handicap, however, for 
there seems to be no reason to impose rigorous laws on the nature of 
learning. Rather, the ultimate purpose of any classification system 
is the need for more information about the learner and tl.e task to be 
learned, and the relationship between these. As long as a taxonomy 
offers some insight to this end, it is a useful classification scheme. 

!7Donald W, Taylor, "Discussion of Papers by Adriaan D. de Groot 
and by Jeffery M, Paige and Herbert A, Simon," Problem Solving; Research , 
Method, and Theory , ed, Benjamin Klelnmuntz (New York, 1966), p. 124. 
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Benjamin Bloom finds that "there could be an almost infinite 
number of ways of dividing and naming the domains of education out- 
comes, since the determination of classes is in some ways arbitrary. 
He suggests a hierarchy of levels of behavior that relate to the diffi- 
culty and complexity of the learning process. ^ Bloom and his asso- 
ciates divided the cognitive domain into six major classes of educa- 
tional objectives, each in turn subdivided into further specific 
behavior classifications. The subdivisions of these major categories 
are vaguely defined, some being distinct in content rather than in 
form. Only by studying the example test items offered can some of the 
ambiguity be cleared away. 

Although we have little difficulty in determining the major 
class within which a behavior falls, we still are not satis- 
fied that there are. enough clearly defined subclassifications 
to provide adequately for the great variety of objectives we 
have attempted to classify. 20 

Bloom’s ii. tention of comprehensiveness, that is, setting up a 
classification scheme which would provide for all cognitive behavior 
found in an educational setting demanded a certain degree of gener- 
ality. The categories Bloom offers present a good generalized model 
of learning activities in the classroom. Because they are so inclusive, 
covering a greater range of educational objectives than is typical at 
the secondary or college level," 21 some alteration of the basic sche me 
is necessary in order to apply it to any specific field. 
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One can be more specific about categorizing objectives by 
narrowing the variety under investigation. f, It is evident that the 
use of a limited number of categories, or components of knowledge or 
skill, does not necessarily imply a limited numb, ’ of kinds of knowl- 
edge or skill * f| 22 what we gain by focusing our attention more 
specifically on the higher level problem-solving categories is a 
classification system appropriate for ordering physics problem com- 
plexity. 

Complexity, to distinguish it from difficulty, is a nonarbi- 
trary objective standard. It is a quality inherent in the task. 
Depending upon one T s qualifications, it may be easy or not to 
achieve. In either case, complexity level can be determined in the 
absence of performance measures and does not require knowledge of 
preceding learning experience* 

Difficulty, on the other hand, is both a more relative and more 
subjective measure. It is often determined by professorial experi- 
ence with prior student performance on similar items, or by analysis 
of student performance, often long after it would be useful for 
formative evaluation. When Bloom says that a reader "must know . . . 

the learning situations which have preceded the test f ,f 23 we are 
reminded that he does not clearly distinguish between difficulty 
and complexity (see n. 19, above). We expect a high correlation 
between these two measures, but we cannot assume their equivalence. 

22 

Francis Mechner , "Behavioral Analysis and Instructional 
Sequencing , " Programmed Instruction , ed. Phil C. Lange (Chicago, 

1967) , p, 83. 

^Bloom Taxonomy , p, 51, 
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By comparing problem complexity with student difficulty, we 
should find evidence of the appropriateness of our hierarchy. "If 
this is the real order from simple to complex, it should be related 
to an order of difficulty such that problems requiring behavior A 
alone should be answered correctly more frequently than problems 
requiring AB . 1,24 It is believed that difficulty, too, follows a 
hierarchal order: 

That Is, the test items lor knowledge of specific facts or 
terms should be passed by more students than those for 
knowledge of rules and principles or skill in the use of 
processes. Also, the test items involving translation and 
application are likely to be more difficult, and thus to 
be passed by fewer students, 25 

If correlations between these two measures are high, it would suggest 
a validity in our taxonomy. 



i 

24 Bloom, Taxonomy . p, 23, 
25Bloom, Handbook , p. 129, 
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Earlier classification systems advanced were inappropriate for 
one or more reasons: too general, covering more kinds of behavior 

than were apparent or necessary to test in a physics problem-solving 
course; too ambiguously or nonobject ively defined, requiring sup- 
positions about internal thought processes or arbitrary assignment 
to subclassiiications ; too history dependent, requiring previous 
learning experiences as a base, For these reasons it was decided 
to adopt an objective measure of problem complex! t> for categoriza- 
tion purposes * This was taken to be the number of computational 
steps required to solve the problem in question; in short, a 
measure of complexity determined by counting mathematical steps* 

For this initial investigation, three categories of objectives 
were identified, and a catch-all gray category was invented* They 
are listed in order of increasing complexity. 

(1) Zero Step Problems: Those questions which do not require 

any mathematical manipulation. Recall of a face or 
definition, or the recognition of an object, fact, or 
definition fall into this category. Even difficult 
conceptual problems or associations are categorized as 
zero step questions whenever there are no algebraic steps 
involved. All word problems; that is, problems which 
are nonnumerical and nonsymbolie , are zero step problems. 

(2) One Step Problems: Those problems which require the 

solution of one algebraic equation for one unknown, or a 
single calculus operation (differential or integration) , 
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(3) Multiple Step Problems: All problems which require the 



solution of more than one algebraic equation, or more 
than one calculus operation. 

We do not distinguish between two, three, or more step problems, 
since we ca: analyze the number of steps only in the intended behavior , 

not the actual behavior, "We do not expect all students to us a the 
skills acquired in a course in exactly the same way. The higher one 
proceeds up the education ladder, the more apparent this becomes. -- 
By eliminating the distinction between, say, two and three step 
problems , we minimize the difference between intended and actual 
behavior , 

In addition, when more than a single operation (step) is required 

to solve a problem, even experts frequently disagree as to the "best" 

way to solve it and the number of steps required. 

Many problems give no clue as to how many, or what, intermediate 
steps must be taken, • • , And if the operation is not a formal 

anal) tic tool, like arithmetic addition, two solvers may apply 
the same operation with different numbers of steps. 27 

Clearly, ambiguities in the step counting process are much more likely 

to occur in multiple step problems, 

A fourth catch-all category included all problems which were 

judged not valid or which did not fall precisely into one of the three 



categories. Problems which had errors, insufficient information, or 
ambiguities were relegated to this category, as were many graphical 
problems which require geometrical steps rather than algebraic steps. 

2^Bloom, Handbook , p. 26. 

-^Helen I, Snyder, Contemporary Educational Psychology: Some 

Models Applied to the School Setting (New York, 1968), p, 66, 
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A majority of problems lacking clear definition required simple 
arithmetic operations; we did not want to equate trivial arithmetic 
computations with algebraic steps. 

Of course,, all the valid problems can still be assigned to 
categories by making the above definitions more exhaustive and 
detailed. More inclusive categories will be developed in the future; 
at present we wish to examine the most elemental scheme without 
including all possible ramifications. 

A complete listing of the items in each category used in the 
analyses can be found in Appendices A, B, and G. 
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The course materials and procedures discussed below describe, 
in part, the physics course delivered under Contract No* 
N0060G-68C-0749 to the U. S, Office of Education for development , 
validation, and installation at the U, S . Naval Academy* The course, 
as delivered, is self-paced, independent study, multimedia, computer 
or manually managed, introductory classical physics. 

There were 513 students available for taking the required course 
at the Academy in the Fall 1969 semester. All had finished one year 
of college at the Academy and were generally engineering and applied 
science majors. Their college experience included one year of 
chemistry and introductory calculus. Understandably, this is a 
highly select and homogeneous group of subjects. The students were 
randomly assigned to one of three groups. Control Group I consisted 
of students taking the conventional course, as it had been developed 
and taught by the Academy Physics staff. Control Group II consisted 
of students also taking the conventional course, but, in addition, 
being given the criterion tests developed for the multimedia course. 

The Experimental Group was made up of those students taking the 
multimedia course developed by N.Y.I.T. under contract to the Office 
of Education. Each week the students were required to complete a 
Study Guide in the form of a scrambled text, which consisted of 45 
to 72 problems. The students responded to these multiple choice 
problems on IBM punch cards, which served as recording and directing 
devices, since the student had to punch his card to find out where to 
go for the next question and to find out if he had made the correct 
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answer selection. As he progressed through the Study Guide, he was 
directed to media presentations, readings, homework assignments, and 
the various other course components. The students worked at their 
own rates throughout the week, but were required to take the criterion 
test together at the end of each week. 

Each student was given a copy of the test, and wrote his answers 
on separate answer forms, which were then scored by comparing 
responses to those listed on a preestablished answer key. In order 
to encourage the students to work consistently throughout the semester, 
it was recommended that the criterion tests be weighted as 60 % of the 
total grade. 

Criterion test questions were prepared from the verbal objectives 
of the development phase of the project by the same team of physicists 
who developed the course materials. The test questions were then sent 
to Annapolis for approval by those members of the Navy Physics staff 
who were involved in the multimedia project. The approved problems, 
with any alterations, additions, or deletions . were then returned to 
the project team at N.Y.I.T., where the questions were analysed and 
assigned to one of three problem categories: zetc step, one step, 

and multiple step. Those problems which could not be uniquely 
classified into one of these categories were assigned to a fourth 
category , gray. Later, if errors which would affect performance were 
found in a problem, if too was reclassified as gray, and excluded 
from the analyses. 

The criterion test for the week's work was given on Saturday 
morning to the midshipmen at the Academy who participated in the 



24 



multimedia course. Tests varied in length from seven to ten problems , 
most having ten. A total of 79 of the 126 problems were acceptably 
classified in this initial effort. 

Those students absent from a group session when the weekly tests 
were given had the opportunity for a make-up exam at a later date. 
Because the exact nature of the malce-up test could not be controlled 
at N.Y.I.T., only those students having taken all 79 criterion test 
questions used for analysis were included in the present study. Of 
197 students in the Experimental Group , only 41 students fulfilled 
this requirement. 

Analyses were conducted on the set of 41 subjects who had 
complete data on all criterion test questions determined to be 
nonarbitrary , The problems were sorted by learning category ? and 
percentage correct in each category was computed for each student. 
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SECTION Vis Calculations and Results 
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Our analysis of the data is directed toward three interrelated 
questions ; 

(1) Is categorizing by complexity consistent with more 
familiar measures of performance? 

(2) What is the relationship between complexity and 
absolute student performance; is complexity a 
good predictive variable? 

(3) What is the relationship between complexity and 
relative student performance; is complexity a 
good predictive variable? 

Intuition dictates fhat problem solving difficulty should be well 
correlated with any reasonable measure of problem complexity. As an 
objective standard of student performance difficulty, we take the 
percentage incorrect for each student in each category. (See 
Appendix D for the individual performance graphs.) By assigning 
numerical measures to the complexity categories; 0, 1, and 2 to 
zero step, one step, and multiple step, respectively, we find the 
relationship between the categories of complexity and student diffi^ 
culty. The correlation coefficient we obtain is 
r = .94 

which is significant at the .01 level. Clearly, difficulty and 
complexity are well correlated, and we conclude that our measure 
of complexity is consistent with the more familiar measure of 
student difficulty. The special advantage, of course, of 
the complexity hierarchy is that it can be determined 
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before testing takes place ; whereas an obj active measure of student 
difficulty can only be obtained after a performance check. 

The data indicate that complexity as defined in this paper is 
an important variable for absolute student performance in problem 
solving. Remarkably, average percentage correct for all students 
in each category (71%, 61%, 49%), is virtually a perfect linear 
function of complexity s as illustrated in Figure 1. The decline of 
absolute performance with increasing complexity is completely 
expected. 




Figure 1 Average percentage correct for all students in each category. 

Correlations of student performance scores for the three pairs 
of categories were calculated. (Figures 2, 3, and 4 represent these 
correlations graphically.) 
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Zero 


step/Multiple step 


r = ,35 


Zero 


step/One step 


r = .44 


One s 


tap /Multiple step 


r - ,65 



The first correlation is significant to the ,05 level; the other 
correlations are significant to the .01 level. 

These correlations show clearly that absolute performance in 
zero step problems is not an accurate predictor of performance in 
one or multiple step problems. Even the higher correlation between 
•- one and multiple step (.65) is insufficient to make a definitive 
prediction of performance in multiple step problems based upon 
performance in one step problems. This conclusion may have important 
consequences for the physics graduate record examinations which are 
generally limited to zero and one step problems. When it is important 
to measure high capabilities for synthesizing complex problem elements, 
multiple step performance must still be tested. 

We note that the correlations indicate a closer relation between 
the two categories which require mathematical manipulation (.65) 
than between the nonmathematical aero step category and the one 
step mathematical category ( .44) . 

A perfect correlation of student performance in any pair of 
categories would indicate that performance does not depend upon com- 
plexity at all since a change in complexity (change in category) 
causes no change in performance. The larger correlations between 
the one step and multiple step categories indicate that performance 
is less sensitive to changes at the high end of the complexity scale. 
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Figure 4 Correlation between one step and multiple step problems. 

To ascertain whether relative performance in one category is 
related to relative performance in other categories, we computed the 
Kendall W. This is a measure of concordance of student rank across 
all three categories , The result is 
W = .63 

which is significant at the .01 level. This agreement between ranks 
is not unexpected. The high correlation indicates that students 
roughly maintain their relative problem solving ability across all 
categories . 

This finding tends to support the altitude that "a 'B' student 
is a ’B* student whatever the test.” The correlation is still too 
low to be taken as anything more than a very rough indicator. In 
short, relative performance in two categories is a crude predictor 
of relative performance in a third category. 
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We have proposed a classification system for problem solving 
behavior based upon complexity. The basic method was to classify 
according to the objective measure of computational steps required 
for the solution of the problem. 

When the categories were assigned a numerical value 5 complexity 
was regarded as a mathematical variable upon which performance was 
strongly dependent * An increase in complexity resulted in a decrease 
in absolute performance. The analysis indicated a closer correlation 
between one and multiple step performance than between zero and one 
or zero and multiple performance. Analysis of concordance of rank 
indicated that students roughly maintain their ranks across com- 
plexity categories. 

The finding of less sensitivity at the complex end of the hier- 
archy further justifies our lack of distinction between different 
numbers of multiple step problems. Originally , we did not distin- 
guish between levels of computational problems above two steps for 
a practical reason: determining how economical a student is in 

solving a multiple step problem is somewhat arbitrary. Experi- 
mental results are consistent with our categorizing these problems 
merely as multiple step. 

Clearly, a prediction of student performance based solely on 
complexity would be questionable, but its utility as an index of 
problem difficulty cannot be minimized. Since a measurable 
objective must specify performance, and performance is highly 
dependent upon complexity, it follows that complexity should be 
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specified for hard science objectives* Even if objectives are stated 

in problem form, the complexity category of the problem should be 

known in order that variations of the problem (for use on a final 

exam, or for comparison purposes) can have equivalent complexity. 

We are finally, slowly, painfully beginning to find better 
ways of measuring educational results , not just in terms 
of achievement in broad areas as measured by standardized 
tests and compared with group norms, but in terms of 
learning outcomes compared with objectives stated in 
behavioral terms , 28 

The taxonomy advanced by this paper should prove to be a useful 
systematizing agent. Defining classes of behavior suggests certain 
implications for the establishment of those behaviors^ which guide 
our instructional procedures. As a result of this investigation, we 
would conclude that enabling objectives should not be more complex 
than the related terminal objective. Enabling objectives should 
facilitate success on the terminal objective; as the complexity of 
an enabling objective increases, it is likely to produce a decrease 
in performance. 

The classification system investigated here presents a useful 
method of categorizing problem solving behavior. It is hoped that 
classification by complexity will contribute somewhat to our under- 
standing of the educational process, 

A more adequate analysis and classification of the variety 
of the processes employed in thinking is prerequisite to the 
development of a more adequate theory of problem solving, 
or of thinking more generally. At this stage, it may well 




^Robert W, Locke, "Has the Education Industry Lost its Nerve?" 
Saturday Review , January 16, 1971, 44, 

^Robert Gagne, "The Analysis of Instructional Objectives for the 
Design of instruction," Teaching Machines and Programmed Learning, II: 
Data and Directions , ed* Robert Glaser (Washington, B ,C , , 1965), p,~25. 
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be most fruitful to focus upon the development of limited 
theories of problem solving* * * - One would hope, however s 

that in the identification of processes in these limited 
areas , one might proceed to the construction of more 
general theories , 30 



30 T ay lor, p, 125. 
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i. At time t = 0 ? a puck is observed to move on a frictionless hori- 
zontal table with a speed of 40 ft/s ec . After two seconds the speed 
f the puck is : 



2. If the work done by a force on an object in moving the object 
through a closed path is zero , the force is called: 

3, f the following statements select the one which does not represent 
one of Kepler's three laws of planetary motion 

A. A line joining any planet to the sun sweeps out equal 
areas in equal times. 

B. The square of the period of any planet about the sun is 
proportional to the cube of the planet's mean distance 
from the sun. 

C. All planets move in elliptical orbits having the sun as 
one focus. 

D. The force of attraction between the sun and each planet 
is along the line joining the two and has magnitude 
which is proportional to the product of their masses and 
inversely proportional to the square of the distance 
between them. 
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4. Experiments performed on the surface of the Earth give a value for 
the universal gravitational constant 

G * 3. <4 X 1Q“0 lb-ft 2 /slug 2 

The mass of the moon is 1,23 v 10~ 2 that of the Earth and its radius is 
0*27 times the Earth’s radius. If an astronaut performed the same exper 
iments on the surface of the moon , what value would he find for G? 



5, From the following expressions select the one in which M m n (or f, M M ) 
stands for inertial mass (as opposed to gravitational mass) . 

A, weight of a body w - GmM/R 2 

B, escape velocity v c «■ /2GM/R 

C, centripetal force F m mv 2 /R 

D * centripetal acceleration a = GK/r 2 

of a satellite 



6- The charge developed on an insulated glass rod J ^d with a silk 
cloth is designated: 
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An electron enters the space between two parallel plates with a hori- 
zontal velocity - 100 km/sec (see diagram above). If the net elec- 
tric field between the plates is |e| = lO^ 1 nt/coul * what will be the 
horizontal velocity of the electron as it leaves the space between the 
plates on the left side? (The charge on an electron is 
q = -1,6 * 10 = ^ coul,) 




In the diagram above, the symbol $ is defined and it is called: 
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Fhich of the following can be considered a Gaussian surf roe? 



(Thtsre may be more than one*) 

A, Spherical shell 

B , Open-ended cylindrical shell 

C, Six-sided cubical shell 

D, A plane 5 3 m x 4 m 



10. The diagram below shows the magnitude of the electric field plotted 
as a function of distance* The dependence of E upon r is given . y the 
equation shown on the diagram. Which of the following objects could 
produce such an electric field? 

A, A uni for ml v charged, non-conducting cylinder 

B A chatted conducting sphere 

C. A charged conducting cylinder 

D. Either B or C 
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In the equation for Gauss's law 



<p 1-dA = -3— 

j 50 

the q term indicates: 

A, The charge enclosed by the Gaussian surface 

B, The net charge enclosed by the Gaussian surface 

C* The net charge enclosed by the Gaussian surface and 

any ot.ier charges n proximity to the Gaussian sur^ 
face 

D, The absolute iue of the net charge enclosed by 
the Gaussian surface 

12. You may need the following constant: 

7 — — = 9 x 10 y nt^m^/coul 2 
4Tre 0 

Choore one or more of the following statements. An electric 
potential, Vg - is : 

A. directly proportional to W^g 

B . indirectly proportional to 

C. indirectly proportional to q Q 

D. directly proportional to 
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I s - Which of the. following is true about' a uniform magnetic field? 
( Hi e r e m ay h e m o r e t h an one . or the r t. m ay b e. none.) 



passing through a magnetic field* 

A charged particle will have a smaller velocity after 
passing through a magnetic field in the direction of 
the field* 

The kinetic energy of an electron cannot be changed by 
a magnetic field regardless of its position or its veloci 



more than one, or the r may he none ) 

A. The magnetic field at points A and B has the same 
magnitude and direction, 

B. The magnetic field at: points A and C are equal in 
magnitude only, 

C* The magnetic field at point D is the largest, 

D, he magnitude of the magnetic field at point C i . 
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the above diagram, which of the following Is true? (Ther may 



greater than the magnitude at point B 
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15. The* MKS "nit of magnetic flux is the 



(one word) , 



16, The diagram below shows the outline of a cathode-ray tube with 
electrons streaming » ..L along the negative x-axis . A coil produces a 
magnetic field B in the positive direction. What must be the direc- 
tion of an electric field in order to cause the electrons to pass 
through the tube undeflected ? 
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stream of electrons 
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». 2 
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i 7 - The loop shown in the figure below would turn n the 
direction , 



T 




18. The diagram shows the elements of a device used ■> measure current. 
The device is ca^ed a (one word). 



Scale 
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19. Which of the following statements are corn :t (there may he more 
than one) , 

A The direction of the magnetic inductic.. lines produced 

by a current-carry* ng conductor is established by making 
use of the right-hand rule; i.e., with the thumb or the 
right hand pointing in the direction of the electron flow, 
the right-hand finper will curl in the samp sense as the 
magnetic induction lines. 

B. The direction of the magnetic induction lines oroduced 
by a current-carrying conductor Is establi. bed by making 
use of the right-hand rule; i.e-, with the thumb of the 
^ight hand pointing in the direction of the current, the 
right-hand finger will curl in the sama sense as the mag- 
netic induction lines. 

C. The magnetic induction lines around a long, straight, 
current-carrying wire are circles whose centers are 
located at the axis of the wire and whose planes are 
normal to the axis of the wir. , 

D. The direction of the magnetic induction lines produced 
by a current-carrying conductor may also be determined 
by making usa of the left-hand rule; i.e., with the 
thumb of the left hand pointing in th' direction o ' the 
current , the left-hand finger will curl around the con- 
ductor in the same sense, as the magnetic induction lines. 
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20. rmuieaxuLcly after the switch in circuit; Is closed the current in 
locp ja will be (select the appropriate one or more) ; 

A* clockwiei^ 

B. counterclockwise 

C. zero 

B, infinite 
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1. The diagram shows how a forr ap 
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S (ft) 
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hid to a 5-lb object varies with 
the displacement of the 
object* Calculate the work 
done by this force in 
moving the object r com the 
origin to s = 10 ft (in- 
clude unit'") . 



2, A constant force of magnitude 100 lb is required to move a block 
along a horizontal floor with constant speed of 4 ft /sec. The force 
is directed along the motion of the block* Calculate the power deliv- 
ered by this force • 



3. 



At point A a .1 00 -kg roller coaster has a kinetic energy equal to 







K = 5000 J 



500^ J; What will be the 
kinetic energy of the 
coaster when it reaches a 
height of 5 m relative to 
the level of A (point B) ? 
(Neglect friction . ) 
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4 = For the system of masses and forces shown in the diagram, the 
acceleration of the center of mass is * 



m , 



4 slug 



F 2 a -601 11) 



30 i lb 






Massless rigid 
connecting rod 



m n 



slug 



5. A 4-kg body is moving toward the positive x-direction with a speed 
ol 1 m/sec. What is the magnitude of the body 5 s momentum? (Include 
units *, ) 



6, Two blocks weighing 4 lb and 2 lb, respectively s rest on a fric- 
tionless lorizontal table* A compressed spring is placed between the 



two blocks but is not attached to either of the blocks, A string tied 
to the blocks keeps them from flying apart. Suddenly ? the string breaks 



4 lb 


i. .i iiA/i i\ unniiiin.fi 


a ib 




y V v y y V2v. .O y “ YJcTjB 


/// ' / / / 



and the 4-lb block is ob- 
served to move to the left 
with a speed of 2 ft/sec. 
The 2-lb block is moving 



to the right with a speed 



of s 
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7. A 3*- kg body Is moving Inward the pos itive x-direction with a speed 
of 2 m/see. An impulsive force applied to this body causes it to change 
its velocity to 5 m/sec toward the positive x- ■■■ Lrect ion , The magnitude 
of the impulse imparted to the body is (include units) : 

8. An impulsive force of constant direction and with average magnitude 
of 500 lb is applied to a body for a duration of 400 m sac 

(1 m sec = 10 -3 sec) , What is the magnitude of the impulse imparted to 
the b dy during this t±rt\e interval? 



9, Two masses 5 m-j = 4 slug and m, ? ^ 2 slug, move toward each other on 
a frictionless table with respective speeds of 4 ft/sec aid 8 ft/sec. 



/> f 7 f / / f f f T 
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They collide and after the 
collision moves directly 
to the left with a speed of 
3 ft/sec* moves toward 
the right and has a speed 
of : 



10. A IQQQ-kg car traveling due east with a speed of 30 m/sec col- 
lides with a 2000-kg truck traveling due north with a speed of 20 m/see . 
Tile two vehicles lock together- The direction of the momentum of the 
two-vehicle body immediately after the collision is: 



O 

ERIC 



5 ? 



